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Abstract 

Online courses are continuing to become an important component of higher education course 
offerings. As the number of such courses increases, the need for quality course evaluations and 
course improvements is also increasing. However, there is not general agreement on the best 
ways to evaluate and use evaluation data to improve online courses. While summative student 
evaluations are commonly used, these may not be the most effective for online course evaluation 
and quality improvement. Formative evaluation is one method of providing course evaluation 
and feedback to the instructor during the course while course improvements can be made to 
benefit the students currently in the course as well as future students. This method of evaluation 
not only provides for an effective course evaluation but also continuous improvement in the 
course. This case study reviews formative evaluations done in two online courses in a Health 
Information Management program. These formative evaluations were found to be quite effective 
for the identification and development of needed course improvement throughout the semester as 
well as for use as input into overall course evaluations. These evaluations demonstrate the value 
of formative evaluations and provide a methodology for implementing such evaluations into an 
online course. 

Keywords: Distance education, course evaluation, quality improvement, self-assessment, 
higher education, feedback, health information management 
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INTRODUCTION 

As the number of online courses continues to grow, the demand for accountability and 
quality in such courses is increasing. One method of improving online courses and contributing 
to a higher level of student learning is through student assessments. There are a wide variety of 
activities that can be completed in online courses to assess quality and student learning and there 
is ample literature on the role of improving student assessments to evaluate student learning. 
However, another method of improving such courses involves the use of innovative course 
evaluation methods. Traditionally, most student course evaluations are summative, completed at 
the end of the semester, used for promotion and tenure, and not always the most useful for course 
improvements. It has been found, however, that formative evaluations, completed during or 
throughout a course may be a better methodology to assess quality and student learning. 
Therefore, there is increased focus on the use of formative evaluations in both face to face and 
online courses. Such assessments, regardless of the methodology used, can lead to immediate 
improvements in the course during the semester in which it is taught. Both students and teachers 
can benefit from such improvements and teaching and learning can become more effective. 

Formative evaluations can be completed in a number of ways, but essentially involve 
some type of course evaluation or student assessment at the midpoint and/or other points 
throughout the course, with fairly immediate feedback. Such feedback can then be used to make 
course changes to provide continuous improvement of the course throughout the semester. This 
benefits the current students and provides the faculty member feedback on teaching methods as 
they are teaching the course. While this is helpful in face-to-face or in online settings, this can be 
especially important in the online setting when student-teacher interaction may be lower than in 
the face-to-face course setting. Often it is more difficult for a teacher in an online class to get the 
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type of feedback that a face-to-face teacher has during classes and in meetings with students. 
Formative evaluation can fit nicely into this gap in online classes as it is defined as evaluation 
“for the purposes of providing feedback to and informing teachers, students, and educational 
stakeholders about the teaching and learning process” (Dunn & Mulvenon, 2009, p. 3). 

Formative evaluation can become a valuable tool for the online course instructor to insure 
continuous improvement in courses and in student learning. This case study was completed to 
evaluate the value and usefulness of formative evaluations in two online courses in the Health 
Information Management Program at a public university in the Midwestern United States. This 
study was used to assess the value of formative evaluations in quality improvement of the 
courses offered as well as to assess how they can serve as a course evaluation tool. 

LITERATURE REVIEW 

With the increased number and popularity of online courses, there is now an increase in 
the demand for accountability and quality of such courses. The number and offering of online 
courses has increased across the country in response to college and university course offering 
decisions, state and national government initiatives, as well as to student demand for this flexible 
learning option. In 2011, Allen and Seaman reported that “over 6.1 million students were taking 
at least one online course during the fall 2010 term; ... and thirty-one percent of all higher 
education students now take at least one course online” (p. 4). With statistics like these it is 
obvious that online courses are a significant part of higher education and need to be included in 
reviews of quality and student learning. However, faculty and administrators have had a difficult 
time discerning how to measure quality in online courses and how to evaluate courses and 
implement course improvements. “Faculty members who teach online cannot apply the same 
instructional techniques...as face-to-face classes” (Thiede, 2012, p. 137). By the same token, 
colleges and universities cannot apply the same evaluative techniques for online classes as they 
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do for face-to-face courses. Pina and Bohn (2014) state that not only do colleges and universities 
do a poor job evaluating online courses, “a contributing factor to this situation may be the 
inability of institutions to adequately measure quality online teaching” (para. 3). Traditionally 
used evaluation techniques may not truly assess course and teaching effectiveness and may not 
be the best tools for online course evaluation. In addition, faculty have found that without tools 
to adequately measure quality online teaching, they cannot determine needs for quality 
improvement of courses to improve student learning. 

The quality of online courses continues to be debated extensively in the literature, with 
little consensus on how to measure it, as well as how to assess teaching effectiveness of 
faculty teaching these courses. One traditional instrument of assessment that has 
historically been utilized to assess students’ perceptions of teaching effectiveness and 
overall course value is the student course evaluation. (Jones, 2012, p. 49) 

Student evaluations of teaching (SETs) are the most common method of assessing 
teachers and courses. These are usually completed at the end of a course or semester and provide 
feedback to the instructor on the course format as well as the teaching methods used. These 
evaluations, however, are commonly used for purposes of promotion and tenure, and, in many 
cases, may not be used directly for course or teaching improvement. “Student course evaluation 
data are not often used for overall quality improvement of teaching, and oftentimes are provided 
to faculty who perhaps do not utilize the information as possible predictors of how to improve 
their teaching effectiveness” (Jones, 2012, p. 51). If they are used by the instructor for 
improvements, those improvements do not benefit the current students, but only students in 
future courses. “Since those students (who) completed such SETs have finished the course..., 
they cannot experience the positive outcome of these evaluations” (Winchester and Winchester, 
2012, p. 672). In some cases, inconsistent course evaluation formats or “course evaluation 
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surveys (that) do not use valid questions or do not have consistent rating approaches” (Bubb, et. 
al., 2013, p. 8), have caused additional problems in the value and use of SETs. Further “all too 
often, summative evaluation ... tells us little more than how popular we are as faculty” (Walker, 
2005, p. 7). The use of summative evaluations may be particularly problematic in online courses 
that may be taught by adjunct instructors yet designed by other faculty. Many online courses are 
based on a master course template which can further complicate selection of evaluation methods. 
While it is clear that course design is an integral part of course quality, the “quality of the 
individual instructor is (also) crucial to the quality of the course” (Pina, Harris, & Ashbaugh, 
2012, as cited in Pina and Bohn, 2014, para 2). While many accrediting bodies are starting to set 
standards for online course quality there is still debate as to how quality should be measured and 
evaluated. Current rubrics for online course evaluation may focus primarily on course design and 
not teaching which does not provide for a fair assessment of teachers teaching courses they 
didn’t design. With all these complicating issues, educators are searching for better ways to 
evaluate online courses that will allow for appropriate evaluation as well as quality improvement. 
One evaluation methodology that is being found to be effective is the use of formative 
evaluations. 

Formative evaluation allows for relatively immediate feedback to the instructor regarding 
the students’ feelings and perceptions on course design as well as teaching. This timely feedback 
during the course enables, “online instructors (to be) able to adjust strategies and methods to 
better meet the needs of current students” (Berridge, Penney, & Wells, 2012, p. 120). This 
creates a “feedback loop for Cl” (continuous improvement) and quality improvement (Aggarwal 
and Lynn, 2012, p. 27). The ongoing use of formative evaluations, and associated course 
improvements result in true Cl, or continuous improvement. “As assessment progresses, the 
learning process is continually monitored and incrementally improved” (2012, p. 29). Formative 
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evaluation has also been seen to increase student participation, encourage student self-evaluation, 
and improve student learning. Bubb, et al, 2013, found that “results obtained by UNLV 
instructors clearly showed that both student learning and classroom performance improve when 
instructors listen to their students’ suggestions and consider implementing them to improve 
teaching effectiveness” (p. 12). 

Formative evaluations can be used to complement the summative, end-of-semester 
evaluations. Such evaluations allow for reflection on the part of the teacher with the opportunity 
for course improvements based on this reflection. Some areas that can be included in the 
evaluations are students’ perceptions of “particular activities, the pace of the delivery and 
success of student engagement. Such information gleaned from a formative SET could allow the 
lecturer to reflect on how to modify the class material for the following week(s) to best meet the 
students’ needs” (Winchester and Winchester, 2012, p. 673). Formative evaluations should 
include questions that are pertinent to both students and teachers and will allow for feedback 
regarding students’ perceptions about the course and teaching methods. If the teacher uses the 
feedback for course improvement, “students will gain additional motivation to complete the 
questionnaires once they see their comments are taken seriously and incorporated in the 
lecturers’ future material” (Winchester and Winchester, 2012, p. 674). The result of such 
evaluations is “greater student appreciation for the opportunity to provide input and instructor 
appreciation for receiving feedback” (Bubb, et. al. 2013, p. 8). Teachers who have used 
formative evaluations have found “greater student performance in ... courses and higher 
summative course evaluation scores” (2013, p. 12). There is an important caveat. “Inherent to 
formative evaluation online is the need to act upon any responses you receive as a class 
instructor, otherwise, you can appear as if you do not care and the evaluation is simply another 
activity students must complete” (Walker, 2005, P. 10). The results of such actions, however, can 
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be used by faculty in their annual evaluations to show course improvements and used by students 
to motivate them to become more involved in the course and to participate more fully. Formative 
evaluations can serve as a method of quality review and evaluation as well as an opportunity for 
faculty to improve the quality of their courses, and thus, student involvement and learning. 

There are a variety of methods that can be used for online course formative evaluations. 
These range from formal peer evaluations, to informal short student surveys. One formal method 
of evaluation is eFACT: Formative Assessment of Classroom Teaching for Online Classes. This 
system “is a process of gathering anonymous student feedback through a faculty consultant using 
e-mail” (Berridge, Penney, and Wells, 2012, p. 120). This anonymous feedback is then provided 
to the instructor for use in changes to the course to improve quality. Some instructors have used a 
modified version of the Face-to-Face Group Instructional Feedback Evaluation Techniques in 
distance education through the use of a facilitator who “can use any number of methods to 
contact students electronically to ask the questions” normally asked in such evaluations (Walker, 
2005, p. 8). Other teachers have used surveys administered to students via SurveyMonkey or, 
most commonly, through online surveys administered through the course software. Most of these 
methods ask students to respond to between three and nine questions regarding what they liked 
or did not like about the course, what helped their learning in the course, what impeded their 
learning in the course, and how the course could be improved. Both closed and open ended 
questions are used in these various methods. Frequency of evaluations can vary from weekly to 
one mid-term evaluation followed by the final, summative evaluation. In order to maximize 
response rate, some instructors will tie participation to grades or to extra credit. Others find that 
students are self-motivated to participate based on the subsequent course improvements. 

Regardless of the method used, teachers have found that formative evaluations provide an 
opportunity to get feedback from students during the course, when changes or improvements can 
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be made to benefit current students. The evaluations provide teachers with the opportunity to 
reflect on their courses and their teaching methods to provide continuous quality improvement of 
their courses. Students benefit from formative evaluations because they can provide feedback 
that can result in fairly immediate changes to the course that can improve their learning and 
increase their motivation to participate in the course. In addition, students are provided “with the 
opportunity for more ownership in the course” (Crews and Curtis, 2011, p. 868). Thus, formative 
evaluations have been found to provide a valuable method of course evaluation for online 
courses. 

METHODS 

For this case study, two online courses were chosen for formative evaluations. These 
courses were both offered in the Health Sciences Department. The first course, Medical 
Terminology, had been offered online for approximately 15 years. The second course, 
Pathophysiology, was being offered online for the first time. Both courses were evaluated using 
online formative evaluations throughout the length of the class. These evaluations were, in turn, 
used for course quality improvement. This case study model was used to evaluate the use and 
value of formative evaluations in online courses. 

Medical Terminology 

Medical Terminology is offered as an on-line only course at this university; there are no 
face-to-face sections offered. It is offered to specific majors during both semesters and the 
summer. These majors are all health related and include nursing, health information 
management, athletic training, and speech pathology. The students in this course can range from 
first semester freshmen working on prerequisites to last semester seniors filling electives. It is 
not known by the instructor at the beginning of each course whether or not the students have 
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taken an online course before or if they are familiar with the university’s learning management 
system. This course, however, does offer some basic instruction on taking courses online and the 
university does provide basic training on the learning management system for students. These 
formative evaluations did not collect information regarding students’ past experiences with 
online courses or the university learning management system. 

At the time of the formative evaluation, the course had been offered in the online format 
for approximately 15 years and offered online-only for approximately eight years. As this course 
had been taught in the same format for many semesters, it was felt that formative evaluation 
could be a useful method to assess the course and make improvements. This course was 
evaluated during the summer of 2014, which was a 6 week compressed version of what is 
normally offered during a regular school semester. There were 28 students in this course and two 
formative evaluations were given. The students consisted of the regular mix of majors and levels 
as outlined above. The instructor administered and analyzed both evaluations. The first was at 
the midpoint of the course and the second was during the final week of the course. In order to 
obtain participation, students were given a small number of extra credit points for completion of 
these evaluations. In addition a standard student course evaluation was given to the students at 
the end of the course. 

The first formative evaluation was provided to the students in the form of an on-line 
minute paper asking the students to respond to four basic questions regarding their opinions of 
the course thus far. The first evaluation consisted of four questions: 1) Name two things about 
this course that you feel are helpful to your learning medical terminology. 2) Name two things 
about this course that you feel are barriers to your learning medical terminology. 3) Name one 
thing the instructor could do to better assist you with learning medical terminology in this course. 
4) Name one thing that you (as a student) could do to better assist you in learning medical 
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terminology in this course. Twenty-one of the twenty eight students responded. Their answers to 
the four questions were analyzed in order to determine potential changes that could improve 
student learning. An action plan was subsequently developed and was implemented for the 
second half of the course. 

The second evaluation was given the last week of the course to determine the success of 
the action plan. The second evaluation consisted of three on-line questions designed to assess the 
students’ perceptions of the changes made after the first evaluation: 1) Did you utilize the 
discussion boards, office hours, or email tips implemented after the Midterm Course Feedback? 
If yes, what tool(s) did you use? 2) If yes, how did you find this to be helpful in your learning of 
medical terminology? 3) If no, why not? The answers to the questions in both evaluations were 
analyzed and used to make changes and improvements to the existing class as well as future 
classes. Nineteen students responded to the final minute paper. The answers to the questions on 
this evaluation were used to assess the success of the changes that had been made to the course 
as well as to assess further changes needed for future offerings of the course. 

In addition to these two on-line minute papers, student scores on the first two exams and 
midterm, before the first evaluation, were compared to student scores on the final two exams and 
the final exam. This allowed for a limited evaluation of student learning changes. 

Pathophysiology I 

Pathophysiology I is the first of a sequence of two required early major courses and is 
one of the courses Health Information Management students take in their first semester of their 
junior year. This course is a content-heavy course with a great deal of knowledge that must be 
learned for subsequent courses, the Registered Health Information Administrator certification 
exam, and students’ future jobs and careers. Pathophysiology is the basis for what sets this major 
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apart from other business majors; the focus on understanding disease and medicine is a 
fundamental component of the major. During the summer of 2014, due to a vacant faculty 
position and the need for release time for a faculty member, the decision was made to change 
Pathophysiology I from a face-to-face course to an online course. This was based on the fact that 
there were no instructors available to teach a face-to-face course during the daytime time slot 
scheduled for the course. However, there was an instructor available to teach an online course. 
This course was also felt to be an appropriate course for the online format due to the fact that it 
was content-heavy and did not involve a high level of critical thinking or the need for lab-based 
experiences. The instructor who had taught the course in the past recorded audio to accompany 
power point slides for all the topics for the course. An adjunct instructor who had prior 
experience teaching online was hired to teach the course in this online format. Due to the short 
time frame surrounding this decision, there was not a formal, well planned transition of this 
course from a face-to-face to an online format. This is recognized as a limitation of this study as 
is the fact that an adjunct instructor taught the course. 

Due to the fact that this was a change in course format, it was decided that formative 
evaluations should be done throughout the Fall 2014 semester to assess the success of the 
changed format and to make needed improvements throughout the course. The principle 
stakeholders were involved throughout this process in order to assess the success (or lack 
thereof) of the course at both the teaching and administration level. Another faculty member, not 
the instructor, administered and analyzed all evaluations. There were 27 students in this course. 
All of these students were junior level Health Information Management majors. It is unknown if 
the students had previous experience with online courses, however, they were all familiar with 
the University’s Learning Management System. There was limited to no instruction on how to 
take the course online. During this semester this required course was only offered online. 
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The first on-line student survey was administered after the first exam. In this class as 
well, students were given a minimal number of extra credit points for completion of each survey. 
The first survey included the following questions: 1) For you personally, what has been the most 
difficult part of the course material so far? 2) What was the most difficult part of the first exam? 
3) What can we do to improve learning outcomes? 

The second on-line survey was administered following the third exam. There were a 
number of course changes that had been made following the first survey, therefore, students were 
asked questions about what changes they had taken advantage of and how those had helped their 
learning: 1) What changes that have been made to the course have you used or taken advantage 
of? 2) If you used any of these changes, how have they helped your learning? 3) What changes 
have you made in your test preparation methods? 4) How have these changes that you have made 
helped you? 5) What areas of the course do you still find difficult? 6) What additional changes 
would you recommend to facilitate your learning? 

Again, in addition to the two student surveys, statistics were gathered on the mean test 
scores for the exams for internal comparison purposes. These were compared to assess learning 
outcomes. 

RESULTS 

Medical Terminology 

The first minute paper in the medical terminology class elicited a 75% response rate 
(21/28 students). The first question asked the students to name two things about this course that 
they felt was helpful to learning medical terminology. The most common answers were the 
practice exams, the chapter content learning modules, and the textbook. Overall the responses 
were positive. Question two asked the students to name two things about the course that they felt 
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were barriers to learning medical terminology. The most commonly listed barrier was that the 
course was online. Related barriers included the fact that there were no lectures or discussions in 
the class and no peer interactions. Students also stated that they felt it was harder to ask questions 
of instructors in online courses and that they had technology concerns. Other barriers listed 
included the fast pace of a summer course as well as the fact that there was a large amount of 
material to learn in the course. The third question asked the students to write one thing that the 
instructor could do to better assist them in learning medical terminology. Interestingly enough, 
the students requested more assignments. They wanted more opportunities to practice the 
terminology and to gain points outside of exams. In addition, they stated that more discussions, 
instructor on-campus office hours, and weekly emails would be helpful. However, many students 
stated that nothing needed to be done to assist them in learning medical terminology. The final 
question asked the students to name one thing they could do to assist themselves in learning 
medical terminology. Most students commented that they needed to stop procrastinating, 
organize their time better, commit time to the course, and to study differently. 

The course grades for the first half of the course were also analyzed for future 
comparison purposes. All 28 students were included in this analysis. It was noted that the course 
grade mean was 88.25%, the mean score on the first two exams was 90.6%, and the midterm 
exam mean was 95.2%. This grade comparison was completed to determine if there were any 
substantive changes in grades before and after the formative evaluation was completed and 
changes were implemented. While many factors could influence grades, it was felt that this 
comparison might elicit additional useful information. 

The second half course grades showed a modest improvement over the first half grades. 
The overall course grade mean and means on the last two exams increased, however the final 
exam mean was slightly lower than the midterm. This can be seen in Table 1. 
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Table 1 


Comparison of First Half Course Grades to Second Half Grades, Medical Terminology 


Grade 

First Half Mean (Standard 
Deviation) 

Second Half Mean (Standard 
Deviation) 

Course Grade 

88.25% (11.49) 

91.12% (18.04) 

2 Exams 

90.6% (13.68) 

92.4% (7.91) 

Midterm/Final 

Exam 

95.2% (18.27) 

89.5% (18.34) 


Interventions for Medical Terminology 

Based on the minute paper responses, the following action plan was put into place. A 
discussion board was added for peer discussion and/or arranging study groups. Two hours of on- 
campus office hour time was added for each week of the final weeks of the course, and weekly 
emails outlining the requirements for the week were also added. An email was sent to all 
students outlining these changes as well as including the following: recommendations for the use 
of the practice exams and chapter content reviews for students who were struggling; comments 
addressing the fast pace and technology concerns for the final weeks of the course; 
encouragement of email questions; and encouragement for students to use good time 
management and study habits for the final weeks of the course. In order to assess the 
effectiveness of these changes, a second minute paper was made available to the students during 
the final days of class. 

Nineteen (68%) students responded to the final minute paper. This three question minute 
paper asked the students basic questions about whether or not they had used the implemented 
improvements, and, if so, if they found them helpful. It was noted that it was apparent that no 
students used the discussion boards or office hours, however, the first question in this minute 
paper asked the students if they had used the discussion boards, office hours, or email tips 
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implemented after the Midterm Course Feedback minute paper. Fourteen students stated that 
they had not used these additions; five stated that they had used the email tips. The second 
question asked the students to note whether or not the improvements that they used had been 
helpful. Four of the five students who had used the email tips stated that they were helpful. The 
third question asked the students to comment as to why they had not used the improvements, if, 
indeed, they had not used them. Fourteen students responded that they were satisfied with their 
study habits or with how they were doing in the class so they saw no reason to use the new tools. 
Two stated that they were not residing close to campus so the office hours were not an option for 
them. Other students stated that they didn’t really think about using the new tools or forgot about 
them. One student stated that they were not aware of the improvements. A review of this 
information indicates that the office hours and discussion boards may be more useful to students 
during the fall or spring semester when more students are on campus and have more time to 
utilize such tools. 

Pathophysiology I 

The first formative evaluation student survey in Pathophysiology, which was given 
following the first class exam and elicited an 85% (23/27 students) response rate, provided a 
great deal of information. The prevailing theme was that the students were not happy with the 
online course format, communication between students and the instructor, grading, being 
required to “teach themselves”, and their inability to assess their own learning prior to the exams. 
Sixty-one percent of the respondents stated that the most difficult part of the course was the 
online format and lack of face-to-face discussion. Student suggestions included more activities, 
quizzes or practice exams so that learning could be assessed prior to the exams (78%). In 
addition, students suggested face-to-face exam preparation sessions that could be used for 
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questions and clarifications (57%). Finally, students requested additional communication from 
the instructor in the form of email responses, updates, and progress reminders (22%). 

These survey results were provided to the instructor, the department chair, and the 
college dean. Following review of the survey results, meetings were held between the 
stakeholders. Weekly face-to-face sessions were added to improve communication between the 
instructor and the students. “Addendum assignments” were added to allow for additional student 
assessment of learning. These were also added to exam scores as extra credit. 

The second online student survey was completed approximately a month later following 
the third exam. This survey also elicited an 85% (23/27) response rate. At the time of this survey, 
the above noted changes had been implemented and student overall course grades had improved 
based on the addendum assignments. The students were much more satisfied with the course 
than they were at the time of the first student survey. However, there were still areas that they 
found difficult or that they felt could be changed to aid in their learning. The vast majority of the 
students had taken advantage of the two main changes that had been made to the course, the 
face-to-face review sessions (59%) and the addendum assignments (82%). The students had 
found that these had improved their learning as well as their focus on the exam material. The 
ability to ask questions and get explanations in person was noted to be helpful. Students also felt 
that the changes had helped them study more effectively and become more confident with the 
material. In addition, the addendums had improved their grades. Students further stated that they 
had revised their test preparation methods to include spending more time preparing (59%), using 
flashcards and quizzing themselves (23%), paying more attention to the notes and audio (14%), 
and becoming more organized (9%). The students stated that they felt that they were more 
prepared and understood the material better and were learning, not just memorizing. Students 
did, however, still find the wording of the test questions and the format of the expected answers 
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difficult (32%). They also cited the large amount of information to be learned (23%), the lack of 
a face-to-face course (18%), and the time allotment for exams to be problematic (9%). They 
requested clearer expectations for answers or the type of exam answers (23%), more review 
sessions that were longer or offered at different times (28%), more assignments (14%), and 
offering the class in a face-to-face setting (14%). 

This report was again forwarded to the instructor, the department chair, and the college 
dean. Only one change was made following the review of this report and that was offering the 
face-to-face review sessions at different times. 

While exam scores are not the sole measure of student learning, they are a method of 
assessing student learning. In order to progress in the Health Information Management major a 
student must obtain a grade of a C or above in this class. The grade scale for the major is also a 
higher scale with a 77% being the lowest C. Therefore, exam scores are a valuable indicator of 
student success in the course and success in moving forward in their major area of study. Table 2 
summarizes the mean exam scores for the first four exams of the course as well as the mean 
exam scores and standard deviations following the addition of the addendums. There was no 
addendum provided for the fourth exam. As can be seen, exam scores did increase over the 
course of the semester after implementation of the above noted changes. The addition of 
addendum assignments to the exams as extra credit dramatically increased scores as well. 

Table 2 

Mean Exam Scores for Pathophysiology Exams 1-4 


Exam 

#1 

Mean (SD) 

#2 

Mean (SD) 

#3 

Mean (SD) 

#4 

Mean (SD) 

Prior to 
addendum 

54.76% 

63.43% 

75.50% 

74.42% 

After 

addendum 

82.10% (15.64) 

89.21% (12.84) 

92.62% (9.84) 

74.42% (13.92) 
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DISCUSSION 

The Medical Terminology formative evaluation project resulted in positive feedback 
regarding the course overall as well as some useful suggestions for improvement. In addition, the 
students stated that they appreciated the initial minute paper and the fact that the instructor was 
interested in their opinions about the class. Most students did not use the improvements, 
however, the course was a short summer course so there was limited time after these were 
implemented. The students did use the email tips, stating that these were helpful. An evaluation 
of the grades before and after the implemented changes did not show any significant change. 
Discussion boards, office hours, and email tips will be included in future Medical Terminology 
courses and further evaluation can be done to see if these are used, and, if so, if they aid in 
student success. Potential long-term plans include adding audio for pronunciation purposes, 
adding additional assignments, and including some student study suggestions in the initial email 
sent to students in future courses. 

The change in the format of the Pathophysiology course affected many on a deep level. 
The students were initially extremely unhappy with this course. There were many student 
complaints. Therefore, the evaluation results were reviewed closely and the student suggestions 
and comments were taken quite seriously. 

The formative evaluations were quite valuable in that they provided the instructor and 
other stakeholders with direct feedback during the course, while changes could be made. They 
enabled a full assessment of what concerns the students had with the course as well as what 
suggestions they had for improvement. These results allowed those involved to take steps to 
improve the course during the semester, resulting in a higher level of student satisfaction and 
greater learning outcomes during the second half of the course. This is also reflected in the 


19 


improved grades as the semester progressed and as changes were made. Based on the students’ 
extreme level of dissatisfaction, it was decided that the second part of the course, 
Pathophysiology II, would be taught in the classroom during the spring 2015 semester. The long 
term use of the formative evaluations in this course was, however, valuable. The stakeholders 
found that the transition of a course to an online format was not as simple as originally thought 
and takes more time than was allotted in this situation. In addition, it was found that student and 
instructor preparation for online courses is of great value in student success and learning. In the 
future, the results of these formative evaluations as well as the lessons learned during this 
process will be used as other courses are transitioned to an online format. 

CONCLUSION 

As the number of online courses continues to grow the need for quality and 
accountability will only increase, therefore, there need to be effective methods of course 
evaluation that evaluate the course and the teaching methodology and provide for improvements 
in these areas as needed. Formative evaluations have been found to play a valuable role in 
ongoing continuous improvement of teaching and learning. “Assessment (or evaluation), in 
general, helps maintain quality and demands accountability” (Aggarwal and Lynn, 2012, p. 44). 
Formative evaluation provides a legitimate method of assessing and improving quality to meet 
the demands for accountability. “Moreover, regardless of the outcome (of formative evaluation), 
students take on a responsible role that encourages them to participate” (Walker, 2005, p. 10). 

The results of the formative evaluations in these two courses clearly shows the value of 
such evaluations throughout a course. The findings in the Medical Terminology course, a long¬ 
time online course, show that there is always room for improvement and making even small 
changes throughout a course can improve student satisfaction and learning outcomes. The 
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findings in the Pathophysiology I course led to more major changes as well as a deeper 
understanding of online course needs for student satisfaction and student learning. Even though 
the students were quite dissatisfied at the beginning of this course, the use of the formative 
evaluations and associated course improvements resulted in a significant increase in student 
satisfaction and learning outcomes. In both courses, students expressed appreciation for the fact 
that their opinions were sought and changes were made based on their input. The fact that their 
opinions were validated and changes were made led many students to feel more a part of the on¬ 
line class experience and increased their role in their success as well. These formative evaluation 
case studies demonstrate that formative evaluations can be a valuable tool in online course 
evaluation and improvement. Formative evaluations provide a method for continuous 
improvement within a course, thus leading to more effective teaching and learning. 



21 


REFERENCES 

Aggarwal, A. K. & Lynn, S. A. (2012). Using continuous improvement to enhance an 
online course. Decision Sciences Journal of Innovative Education, 10, 25-48. 
doi: 10.1111/j. 1540-4609.2011.00331 .x 

Allen, I.E. & Seaman, J. (2011). Going the distance: Online education in the United 
States, 2011. Retrieved from Babson College website: 
http://www.babson.edu/Academics/centers/blank-center/global- 
research/Documents/going-the-distance.pdf 

Berridge, G. G., Penney, S., & Wells, J. (2012). eFACT: Formative assessment of classroom 

teaching for online classes. Turkish Online Journal of Distance Education, 13, 119-130. 
Retrieved from http://files.eric.ed.gov/fulltext/EJ976930.pdf 

Bubb, D. K., Schraw, G., James, D. E., Brents, B. G., Kaalberg, K. F., Marchand, G. C., Amy, 
P., Cammet, A. (2013). Making the case for formative assessment: How it improves 
student engagement and faculty summative course evaluations. Assessment Update, 25, 
8-12. doi:10.1002/au 

Crews, T. B. & Curtis, D. F. (2011). Online course evaluations: Faculty perspective and 

strategies for improved response rates. Assessment & Evaluation in Higher Education, 
36, 865-878. doi: 10.1080/02602938.2010.493970 

Dunn, K. E. & Mulvenon, S. W. (2009). A critical review of research on formative assessment: 
The limited scientific evidence of the impact of formative assessment in education. 
Practical Assessment, Research & Evaluation, 14, 1-11. Retrieved from 
http ://pareonline .net/get vn. asp? v= 14&n=7 


22 


Jones, S. J. (2012). Reading between the lines of online course evaluations: Identifiable actions 
that improve student perceptions of teaching effectiveness and course value. Journal of 
Asynchronous Learning Networks, 16, 49-58. Retrieved from 
http ://files. eric. ed. go v/fulltext/EJ971039.pdf 

Pina, A. A. & Bohn, L. (2014). Assessing online faculty: More than student surveys and design 
rubrics. Quarterly Review of Distance Education, 15, 25-34. 

Thiede, R. (2012). Best practices with online courses. US-China Education Review, 2, 135-141. 
Retrieved from http://files.eric.ed.gov/fulltext/ED532176.pdf 

Walker, S. L. (2005). Modifying formative evaluation techniques for distance education class 
evaluation. Turkish Online Journal of Distance Education, 6, 7-11. Retrieved from 
http://tojde.anadolu.edu.tr/yonetim/icerik/makaleler/212-published.pdf 

Winchester, M. K. & Winchester, T. M. (2012). If you build it will they come? Exploring the 

student perspective of weekly student evaluations of teaching. Assessment & Evaluation 
in Higher Education, 37, 671-682. doi: 10.1080/02602938.2011.563278 



23 


Appendix A 

Medical Terminology Formative Evaluations 

Formative Evaluation #1 (online minute paper administered through learning management 

system at midpoint in course) 

1. Name two (2) things about this course that you feel are helpful to your learning 
medical terminology. 

2. Name two (2) things about this course that you feel are barriers to your learning 
medical terminology. 

3. Name one (1) thing the instructor could do to better assist you with learning medical 
terminology in this course. 

4. Name one (1) thing that you (as a student) could do to better assist you in learning 
medical terminology in this course. 

Formative Evaluation #2 (online minute paper administered through learning management 

system during final week of course) 

1. Did you utilize the discussion boards, office hours, or email tips implemented after 
the Midterm Course Feedback? If yes, what tool(s) did you use? 

2. If yes, how did you find this to be helpful in your learning of medical terminology? 


3. If no, why not? 
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Appendix B 

Pathophysiology I Formative Evaluations 

Formative Evaluation #1 (online minute paper administered through learning management 
system at first month point in course) 

1. For you personally, what has been the most difficult part of the course material so 
far? 

2. What was the most difficult part of the first exam? 

3. What can we do to improve learning outcomes? 

Formative Evaluation #2 (online minute paper administered through learning management 
system at two month point in course) 

1. What changes that have been made to the course have you used or taken advantage 
of? 

2. If you have used any of these changes, how have they helped your learning? 

3. What changes have you made in your test preparation methods? 

4. How have these changes that you have made helped you? 

5. What areas of the course do you still find difficult? 

6. What additional changes would you recommend to facilitate your learning? 



